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FOREWORD 


The  Presidio  of  Monterey  Field  Unit  has  an  ongoing  program  of 
research  on  Army  unit  training.  One  facet  of  this  research  is  an  exam¬ 
ination  of  why  and  how  some  training  innovations  are  used  by  operational 
units  while  others  are  ignored.  The  problem  of  non-use  is  Army-wide  and 
has  been  called  the  "implementation  gap." 

At  root,  implementation  gaps  exist  because  of  a  lack  of  communi¬ 
cation  between  the  training  developer  and  the  operational  units  which 
are  expected  to  use  the  innovation.  All  training  innovations  require 
time  for  units  to  study  the  innovation,  understand  its  advantages  and 
where  it  fits  into  unit  training,  decide  what  resources  it  requires,  and 
to  incorporate  it  fully  into  the  routine  training  system.  During  this 
time,  even  the  best  training  innovation  needs  support,  and  providing, 
planning,  and  orchestrating  this  support  requires  the  cooperation  of  the 
developer,  MACOM  headquarters,  and  user  units. 

This  guide  provides  a  roadmap  to  help  TRADOC  developers  and  the 
headquarters  of  MACOMS  plan  the  implementation  of  a  training  innovation. 
When  used  before  the  innovation  is  fielded,  the  guide  will  lead  to  a 
consideration  of  the  problems  the  innovation  will  encounter  and  to  the 
development  of  strategies  or  support  needed  to  overcome  those  problems. 
The  goal  is  to  support  the  training  innovation  through  the  critical 
fielding  phase  until  it  achieves  the  status  of  routine  and  effective 
use. 


Fuller  discussion  of  the  "implementation  gap"  problem  is  provided 
by  ARI  Research  Report  1344  entitled  On  Closing  the  Implementation  Gap: 
Symposium  Proceedings  (in  press). 


EDGAR  M.  JOHNSON 
Technical  Director 
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EXECUTIVE  SUMMARY 

A  GUIDE  TO  IMPLEMENTATION 
OF  TRAINING  PRODUCTS 


Requirement 

This  guide  describes  a  strat¬ 
egy  for  implementing  new  training 
products  in  US  Army  units.  Its 
purpose  is  to  help  the  training 
product  manager  ensure  that  opera¬ 
tional  units  use  and  sustain  new 
products  once  they  are  introduced. 
This  guide  will  assist  the  product 
manager  in  planning,  executing  and 
monitoring  an  implementation  effort 
that  begins  when  the  product  is 
developed  and  ends  when  it  is  fully 
integrated  into  training  in  units. 

The  guide  was  developed  because 
there  is  little  information  available 
at  present  on  how  to  implement  new 
training  products  to  ensure  that 
they  are  used  as  intended.  The 
Army  already  has  methods  for  devel¬ 
oping  a  product  (see  US  Army 
Training  and  Doctrine  Command, 

TRADOC  Pamphlet  350-30  Interservice 
Procedures  for  Instructional  Systems 
Development,  for  example),  for 
using  a  product  in  unit  training 
(e.g.,  Training  Circulars  and 
Training  Manuals),  and  for  evalua¬ 
ting  product  effectiveness  (e.g., 
TRADOC  Pamphlet  71-8  Analyzing 
Training  Effectiveness).  This 
guide  bridges  the  gap  between 
development  on  the  one  hand  and  use 
and  evaluation  on  the  other. 

Without  efforts  by  product  managers 
in  this  critical  area,  a  demonstra¬ 
bly  effective  training  innovation 
may,  after  delivery  to  the  field, 
never  be  used. 


Product: 

The  guide  was  developed  based 
on  studies  conducted  during  an 
evaluation  of  the  implementation  of 
MILES,  the  Multiple  Integrated 
Laser  Engagement  System.  This 
Tactical  Engagement  Simulation 
(TES)  was  expected  to  face  a  wide 
range  of  implementation  problems; 
accordingly,  a  monitoring  effort 
was  begun  to  track  the  progress  of 
MILES  fielding  and  to  identify  any 
implementation  difficulties. 

During  1979-80  a  model  entitled 

A  Strategy  for  Coping  With  Change 
in  Army  training  was  developed. 

The  strategy  was  then  translated 
into  the  present  practical  guide  to 
assist  product  managers  in  the 
implementation  of  training  products 
which  are  to  be  exported  to  opera¬ 
tional  units. 

Um: 

The  guide  first  provides 
procedures  for  doing  an  implementa¬ 
tion  analysis  to  identify  problems 
likely  to  be  encountered  during 
implementation.  It  then  presents  a 
method  for  planning  an  implementa¬ 
tion  effort  to  integrate  a  specific 
product  into  a  particular  training 
environment. 
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A  GUIDE 

TO  IMPLEMENTATION 
OF  TRAINING  PRODUCTS 


INTRODUCTION 

THE  NEED  FOR  A  GUIDE  TO 
IMPLEMENTATION 


I 


The  United  States  Army  develops,  imple¬ 
ments  and  uses  its  own  training  products.  This 
life-cycle  control  is  an  advantage,  but  some 
problems  occur  in  meshing  the  efforts  of 
developers,  such  as  US  Army  Training  and 
Doctrine  Command  (TRADOC)  and  US  Materiel 
Development  and  Readiness  Command 
(DARCOM),  user  units,  and  Operational  Major 
Army  Command  (MACOM)  sponsors.  Each 
command  is  primarily  concerned  with  its 
mission,  and  often  a  gap  occurs  between 
product  hand-off  and  product  use.  For  example, 


a  developer  may  send  a  high-quality  but 
difficult-to-use  product  to  the  field.  Users  try  to 
continue  training  and  can't  find  time  to  gear-up 
for  the  new  product.  Unused  products  begin  to 
stack  up  in  user  units,  and  an  implementation 
gap  opens. 

This  guide  is  designed  for  the  product 
manager  of  a  new  training  product.  It  presents 
a  procedure,  not  only  for  issuing  a  product  to 
the  field,  but  also  for  ensuring  that  it  is  used 
as  intended. 


IMPLEMENTATION 
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DEVELOPER’S 
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IMPLEMENTATION 
FROM  THE  USER’S 
POINT  OF  VIEW 


THE  BENEFITS  OF  A  GUIDE  TO 
IMPLEMENTATION 


The  bottom  line  of  training  product  imple¬ 
mentation  is  improved  training  in  operational 
units  All  other  activities  are  means  to  this  end. 
This  guide  will  further  good  training  in  three 
ways 

It  will: 

•  FOCUS  ON  USE  AND  SUSTAINMENT 
OF  TRAINING  PRODUCTS  IN  USER 
UNITS.  The  purpose  of  every  recommended 
action  and  guideline  is  to  increase  the 
operational  unit’s  ability  to  accept  operate, 
manage  and  control  a  new  training  product 

•  ESTABLISH  PROCEDURES  TO  CO¬ 
ORDINATE  THE  EFFORTS  OF  ALL  THE 
MAJOR  AGENCIES  IN  THE  IMPLE¬ 
MENTATION  PROCESS.  Included  here 
are  formal  planning  and  negotiation  steps 
which  will  bridge  the  efforts  of  TRADOC, 
DARCOM  and  other  MACOMs  concerned 
with  implementation  of  training  products 
The  guide  first  becomes  useful  during  the 
full-production  and  deployment  phase  of 
the  Life  Cycle  System  Management  Model 
for  Army  Systems  (Department  of  the  Army 
Pamphlet  1 1  -25)  when  users  need  to  be 
informed  that  a  product  is  on  its  way  to 


the  field.  The  steps  in  this  guide  correspond 
to  the  Implementation  and  Control  Stages 
of  the  Interservice  Procedures  for  Instruc¬ 
tional  Systems  Development  Model 
(TRADOC  Pamphlet  35030)  for  introduc¬ 
ing  training  products  into  TRADOC  resident 
training  programs.  This  guide  is  tailored 
specifically  to  introducing  and  sustaining 
exportable  training  products  in  operational 
units. 

.  BUILD-IN  FEEDBACK  TO  CONTROL 
IMPLEMENTATION.  Some  problems  — 
ammunition  shortages,  maintenance 
requirements  etc.  —  can  be  predicted  in 
advance.  Other  problems  can  only  be 
detected  by  monitoring  product  use  in 
units  It  is  impossible  to  predict  for  example, 
how  individuals  will  react  to  a  new  product 
whether  or  not  they  will  dislike  the  product 
for  a  non-rational  reason.  No  one  knows 
the  overall  effects  on  the  training  system 
as  a  whole;  perhaps  the  product  will  drain 
resources  from  other  vital  activities  Con¬ 
tinued  monitoring,  using  the  procedures 
in  this  guide,  can  alert  the  product  manager 
to  problems  which  arise  as  the  product 
becomes  integrated  into  Army  training 


This  guide  is  intended  for  you,  the  persons 
or  agencies  responsible  for  implementing 
exportable  training  products  for  US  Army  training. 
The  guide  organizes  your  effort  into  four  major 
blocks  as  shown  in  Figure  1 . 


BLOCK  1 


BLOCK  2 


Perform  an  implementation  analysis  to  determine  If  an  implementation  effort  is  necessary,  select 


what  effort  is  necessary  to  get  your  product 
used  First  conduct  a  training  product  analysis; 
then  use  the  product  analysis  to  identify 
potential  implementation  problems 


implementation  strategies,  develop  a  plan  for  an 
implementation  program,  and  develop  a  plan 
for  monitoring  and  feedback. 


Throughout  this  guide  worksheets  are 
provided  to  help  you  develop  and  carry  out  a 
successful  implementation  program  for  your 
particular  training  product  These  worksheets  are 
not  highly  detailed,  because  different  kinds  of 
preparations,  execution  actions  and  success 
indicators  will  apply  for  implementation  of 
different  kinds  of  products  The  worksheets  are 
designed  so  that  you  can  use  your  knowledge 
of  the  product  to  fill  in  the  proper  level  of  detail. 


Throughout  the  rest  of  the  guide.  Figure  1  is 
reproduced  in  miniature  with  appropriate  boxes 
highlighted  to  serve  as  a  roadmap  and  reference 
guide. 


FIGURE  1 

OVERVIEW 

A  GUIDE  TO  IMPLEMENTATION 
OF  TRAINING  PRODUCTS 


BLOCK  3 

Execute  an  implementation  program.  Such  a 
program  is  composed  of  four  stages:  1)  Orienta¬ 
tion,  2)  Fielding,  3)  Trial,  and  4)  Integration.  Each 
stage  has  a  preparation,  execution,  and  success- 
indicator  component 


BLOCK  4 

Conduct  monitoring  and  feedback  during  the 
Implementation  Program:  diagnose  problems 
and  achievements,  and  provide  feedback 
to  appropriate  action  agencies 


BLOCK  1 

IMPLEMENTATION  ANALYSIS 


IMPtS  MENTATION 
ANAJ.YBJS 


IMPLCMKNTATTON 


MONITOMNO 


G5ci 


If  you  have  a  training  product  to  export, 
an  implementation  analysis  will  estimate  what 
kind  of  effort  will  be  needed  to  get  it  used. 

An  implementation  analysis  has  two  steps  as 
shown  in  Figure  2. 

•  CONDUCT  A  TRAINING  PRODUCT 
ANALYSIS:  Clearly  describe  the  features 
of  the  product  which  are  important  to 
implementatioa 

•  IDENTIFY  POTENTIAL  IMPLEMENTA¬ 
TION  PROBLEMS:  Analyze  how  well  the 
product  is  going  to  fit  into  training  given 
1)  the  kind  of  soldiers  who  will  use  and 
manage  it  and  2)  the  kind  of  units  to  which 
it  is  being  sent 

A  good  implementation  analysis  will  put 
your  implementation  program  right  on  target 
It  will  direct  attention  to  the  real  problems  and 
provide  a  convenient  checklist  to  see  that  no 
important  details  are  overlooked  Lets  now  look 
at  each  step  in  the  Implementation  Analysis 
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FIGURE  2 

BLOCK  1 

IMPLEMENTATION  ANALYSIS 


BLOCK  1 

IMPLEMENTATION  ANALYSIS 


STEP  1 


STEP  2 


FIGURE  3 

WORKSHEET  FOR 
TRAINING  PRODUCT  ANALYSIS 


CONDUCT 

TRAINING  PRODUCT  ANALYSIS 
Use  OT/DT  results,  FEAs,  concept 
studies,  etc.  to  identify  these 
features  of  the  product 


Define  expected  utility: 


1.  Performances  that  will  be  improved: 


2.  Training  costs  that  will  be  reduced: 


3.  Improvements  in  evaluation  and  feedback 


4.  Improvements  in  delivery  of  training: 


Identify  product  overhead  (I.e.,  cost  to  use): 


1.  Training  and  support  personnel: 


2.  Time  for  set  up  use  take-down: 


3.  Facilities  for  use  storage  and  maintenance: 


4.  Accountability  and  record-keeping  requirements: 


5.  Consumable  supplies  required  to  support  proper 
use  (eg,  batteries): 


Identify  skill  and  information  requirements: 


1.  Skills  users  must  possess  to  use  and  maintain: 


2.  Information  that  users  must  have  to  use  product 
effectively. 


Define  developer's  concept  of  product  use: 


1.  Location  where  it  will  be  used  (schools  or  units? 
field  or  classroom?): 


2.  Type  of  training  it  will  support  (collective  or 
individual?  for  which  individuals  or  units?): 


3.  Where  it  will  fit  in  the  training  system.  (e.g,  training 
support  package  for  a  weapons  system,  or  a 
product  to  support  the  training  system  or  a  complete 
training  system): 


4.  Expected  frequency  of  use: 
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IMPLEMENTATION  ANALYSIS 
IDENTIFY  POTENTIAL 
IMPLEMENTATION  PROBLEMS 

STEP  2 


ACCEPTANCE: 

GIVEN  CURRENT 
PRIORITIES  AND 
COMMAND  EMPHASIS, 
WILL “EXPECTED” 
UTILITY  GAIN  READY 
ACCEPTANCE? 


KNOW-HOW: 

GIVEN  AVAILABLE 
“INFORMATION  AND 
TRAINING  SKILLS,” 

DO  PERSONNEL  HAVE 
THE  KNOW-HOW  TO 
USE  THE  PRODUCT? 


MANAGEMENT: 

GIVEN  CURRENT 
RESOURCE  AVAILABILITY 
WILL  “OVERHEAD”  BE 
EASILY  ACCOMMODATED 
IN  TRAINING 
MANAGEMENT? 


POLICY: 

GIVEN  CURRENT 
TRAINING  GUIDANCE 
AND  REGULATIONS, 
WILL  “PRODUCT  USE" 

BE  EASILY  INTEGRATED 
INTO  TRAINING  POLICY? 


New  products  can  cause  problems  for 
individual  users  and  for  the  operational  units 
by  making  demands  on  people's  time  and  units’ 
resources  Identifying  potential  implementation 
problems  will  determine  how  difficult  imple¬ 
mentation  is  likely  to  be 

•  ANALYZE  FIT  BETWEEN  PRODUCT 
AND  INDIVIDUAL  USERS.  Based  on  the 
training  product  analysis  (Pg.  9)  and  your 
knowledge  of  the  training  environment 
determine  whether  your  product  will  be 
readily  accepted  and  whether  user  person¬ 
nel  have  the  know-how  to  use  the  product 

GIVEN  CURRENT  PRIORITIES  AND 
COMMAND  EMPHASIS  WILL  THE 
“EXPECTED  UTILITY"  GAIN  READY 
ACCEPTANCE? 

GIVEN  AVAILABLE  “INFORMATION  AND 
TRAINING  SKILLS,”  DO  PERSONNEL 
HAVE  THE  KNOW-HOW  TO  USE  THE 
PRODUCT? 


•  ANALYZE  FIT  BETWEEN  PRODUCT 
AND  USER  ORGANIZATIONS.  Based  on 
the  training  product  analysis  (Pg  9)  and  your 
knowledge  of  the  training  environment 
determine  whether  user  units  can  manage 


the  product  overhead  and  regulate 
product  use  within  the  framework  of  current 
policies 

GIVEN  CURRENT  RESOURCE  AVAILA¬ 
BILITY  WILL  “OVERHEAD"  BE  EASILY 
ACCOMMODATED  IN  TRAINING 
MANAGEMENT? 

GIVEN  CURRENT  TRAINING  GUIDANCE 
AND  REGULATIONS,  WILL  PRODUCT  USE 
BE  EASILY  INTEGRATED  INTO  TRAINING 
POLICY? 

If  additional  information  about  the  current 
training  environment  is  needed,  ask  people  in  the 
operational  units  about  priorities  and  command 
emphasis,  and  collect  from  these  units  available 
information  on  training  skills  resource  availability 
and  training  guidance  and  regulations 

If  you  find  any  cases  where  the  intended 
users  do  not  value  the  product’s  expected 
utility  (e.g,  training  benefits)  or  where  users  can 
not  operate,  afford  or  regulate  the  product  you 
have  discovered  a  potential  implementation 
problem.  By  continuing  through  this  guide,  you 
can  develop  a  program  to  prevent  these 
problems  from  occurring  or  at  least  to  keep  track 
of  them  so  you  will  know  if  your  product  gets 
into  trouble.  Completing  a  worksheet  such  as  the 
one  shown  in  Figure  4  will  help  you  to  identify 
potential  implementation  problems 


FIGURE  4 

WORKSHEET  FOR  IDENTIFYING 
POTENTIAL  IMPLEMENTATION 
PROBLEMS 


IDENTIFY  POTENTIAL 
IMPLEMENTATION  PROBLEMS 
Using  results  of  training  product 
analytes  ( pg.  9)  and  your  knowledge 
of  the  user  environment, 
answerthese  questions 


KNOW-HOW:  Given  available  “Information 
and  training  akllla,”  do  personnel  have  the 
KNOW-HOW  to  uae  the  product? 


ACCEPTANCE:  Give  current  priorities  and 
command  emphasis  will  “expected  utility"  gain 
ready  ACCEPTANCE? 


□  YES  □  NO 

If  no,  what  is  the  problem? 

For  training  managers: 


For  trainers: 


MANAGEMENT:  Given  current  resource  availa¬ 
bility  will  "overhead”  be  easily  accommodated  in 
training  MANAGEMENT? 

a  YES  □  NO 

If  no,  what  additional  resources  do  they  need? 
Personnel: 

Time: 


Space: 


Funds: 


Consumables: 


□  YES  □  NO 

If  no.  what  special  training  or  instruction  is  needed? 
For  training  managers: 


For  trainers: 


For  trainees: 


POLICY:  Given  current  training  guidance  and 
regulations  will  “product  use”  be  easily  integrated 
into  training  POLICY? 

□  YES  □  NO 

If  na  cite  regulations  which  conflict  with  implementation 
envisioned 


BLOCK  2 

IMPLEMENTATION  PLANNING 


Once  the  need  for  an  implementation 
program  is  identified,  it  is  time  to  pian.  Imple¬ 
mentation  Planning  has  three  basis  steps  as 
shown  in  Figure  5. 

•  SELECT  IMPLEMENTATION 
STRATEGIES:  Not  all  implementation 
problems  can  be  solved  in  the  same  way. 
Different  problems  require  different 
strategies.  There  are  two  fundamental  kinds 
of  strategies  to  choose  from:  enabling 
strategies,  which  educate  and  assist 
with  the  product  use,  and  incentive  strate¬ 
gies,  which  motivate  and  regulate  product 
use 

•  DEVELOP  A  PLAN  FOR  THE  IMPLE¬ 
MENTATION  PROGRAM:  A  good 
implementation  plan  1 )  determines  the 
sequence  or  stages  in  the  proposed 
program,  2)  specifies  the  actions  to  be  taken 
during  each  stage  Le,  the  execution  of  the 
program,  3)  assigns  responsibility  for 
actions  to  developers,  users  and  supporters 
of  the  product  and  4)  specifies  desired 
outcomes  or  success  indicators  for  the 
program. 


.  DEVELOP  A  PLAN  FOR  MONITORING 
AND  FEEDBACK:  Monitoring  and  feed¬ 
back  keep  an  implementation  program  on 
track  and  make  timely  adjustments  possible 
A  plan  for  monitoring  and  feedback 
1 )  identifies  agencies  needing  information 
about  progress  of  the  implementation 
program,  2)  develops  a  schedule  for 
monitoring  and  feedback  and  3)  identifies 
potential  information  sources  and  informa¬ 
tion  collection  strategies 

The  following  pages  contain  instructions 
and  worksheets  for  developing  an  implementa¬ 
tion  plan  for  Army  exportable  training  products 
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FIGURE  5 

BLOCK  2 

IMPLEMENTATION  PLANNING 


BLOCK  2 

IMPLEMENTATION  PLANNING 


STEP  1 


STEP  2 


STEP  3 
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There  are  two  fundamental  kinds  of  imple¬ 
mentation  strategies,  each  of  which  divides  into 
two  parts,  as  shown  above 

.  ENABLING  STRATEGIES  make  a  user 
capable  of  using  a  product 

-  EDUCATE:  Retrain  or  provide  information 
which  will  enable  people  to  use  the 
product 

-  ASSIST:  Provide  any  necessary 
resources  or  services  which  are  not 
ordinarily  available  to  the  user  organizatioa 


The  right  strategy  depends  on  the  type  of 
problem  you  have.  It  doesn’t  help  to  motivate 
people  and  get  them  excited  about  a  product 
if  they  lack  the  skills  to  use  it  Nor  is  it  wise 
to  field  expensive,  complex  technology  if  the 
Army  is  not  going  to  regulate  its  use. 

Completing  a  worksheet  such  as  is  shown 
in  Figure  6  will  help  you  to  select  implementation 
strategies 


•  INCENTIVE  STRATEGIES  make  per¬ 
sonnel  and  units  more  likely  to  accept  and 
use  the  product 

-  MOTIVATE:  Persuade  soldiers  and  leaders 
to  believe  in  the  product  and  make  them 
want  to  use  it  properly. 

-  REGULATE:  Control  the  activities  of  indi¬ 
viduals  and  units  through  policies 
regulations  and  doctrine  to  ensure  high 
quality  use  of  the  product 


FIGURE  6 

WORKSHEET  FOR  SELECTING 
IMPLEMENTATION  STRATEGIES 


Using  the  list  of  potential  implementation  problems  you  have  identified  (pg.  11), 

answer  this  question  - 

WHAT  KINO  OF  IMPLEMENTATION 
PROBLEM  HAVE  YOU  IDENTIFIED? 


ACCEPTANCE 

Which  solution  do  you  think  Is  host  for  GAINING 
ACCEPTANCE  OF  THE  PRODUCT? 

Increase 

Persuade  key 

Have  MACOMs 

product  utility 

users  they 

require  use  of 

with  further 

really  need 

the  product 

development 

the  product 

List  problem 

List  problem 

List  problem 

and  potential 

and  potential 

and  potential 

solution  in 

solution  in 

solution  in 

REGULATE 

MODIFY 

MOTIVATE 

box 

PRODUCT  box 

box 

MANAGEMENT 

Which  solution  do  you  think  is  best  for  ENSURING 
ADEQUATE  MANAGEMENT  OF  OVERHEAD? 

Reduce 

Modify 

Provide 

resource 

existing 

additional 

demands  and 

training 

resources 

management 

management 

rqts  of  product 

procedures 

List  problem 
and  potential 

List  problem 

List  problem 

solution  in 

and  potential 

and  potential 

ASSIST  box 

solution  in 

solution  in 

MODIFY 
PRODUCT  box 

ASSIST  box 

KNOW-HOW 

Which  solution  do  you  think  is  best  to  ENSURE 
ADEQUATE  KNOW-HOW? 

Change  pro- 

tnform/train 

Provide 

duct  to  simplify 

users  in  new 

special  skills 

user  task  rqts 

skills 

thru  outside 
agencies 

List  problem 

List  problem 

and  potential 

and  potential 

List  problem 

solution  in 

solution  in 

and  potential 

MODIFY 

EDUCATE  box 

solution  in 

PRODUCT  box 

ASSIST  box 

POLICY 

Which  solution  do  you  think  is  best  for 
INTEGRATING  PRODUCT  INTO  POLICY? 

Adjust 

Modify 

Develop 

recommended 

existing 

new  training 

concept  of 

training 

guidelines 

use 

policies 

List  problem 

List  problem 

List  problem 

and  potential 

and  potential 

and  potential 

solution  in 

solution  in 

solution  in 

REGULATE 

MODIFY 

REGULATE 

box 

PRODUCT  box 

box 

MODIFY  PRODUCT 


EDUCATE 


ASSIST 


MOTIVATE 


REGULATE 


IMPLEMENTATION  PLANNING 

DEVELOP  A  PLAN  FOR 
IMPLEMENTATION  PROGRAM 

STEP  2 


Until  now  this  guide  has  preceded  step 
by  step  with  the  reader  reading  the  left-hand 
page  and  filling  out  the  right  hand  worksheet. 
At  this  point  the  procedure  changes.  Although 
we  are  still  in  the  planning  portion  of  the 
implementation  process,  this  step  consists 
of  filling  out  worksheets  which  will  be  used 


in  Block  3  (Implementation  Program). 
Completing  the  worksheets  according  to 
instructions  on  these  two  pages  (Pg.  1 6, 1 7) 
will  provide  you  with  a  plan  to  be  used  during 
the  Implementation  Program  (Block  3).  Fill 
out  the  worksheets  from  pages  25-53  now 
and  use  them  later. 


DEVELOP  A  PLAN 
FOR  IMPLEMENTATION 
PROGRAM 


SEQUENCE  OF 
IMPLEMENTATION 


ACTIONS 
TO  8E  TAKEN 


RESPONSIBILITIES  OF 
DEVELOPERS.  USERS 
AND  SUPPORTERS 


DESIRED 

OUTCOMES 


STAGES  OF 
IMPLEMENTATION 
PROGRAM 
pp  23,  31 , 39,  47 


EXECUTION  BLOCKS 
OF  IMPLEMENTATION 
PROGRAM 
pp  27,  35,  43,  51 


PREPARATIONS  BLOCKS 
OF  IMPLEMENTATION 
PROGRAM 
pp  25,  33,  41,  49 


SUCCESS 

INDICATION  BLOCKS 
OF  IMPLEMENTATION 
PROGRAM 
pp  29.  37.  45,  53 


The  parts  of  a  plan  for  an  implementation 
program  are  shown  in  Figure  7. 

.  THE  SEQUENCE  OF  IMPLE¬ 
MENTATION:  The  implementation  section 
of  this  guide  breaks  implementation  into 
four  stages:  1)  Orientation  —  the  user  finds 
out  about  a  product  which  is  coming; 

2)  Fielding  —  the  product  arrives;  3)  Trial  — 
the  user  tries  the  product  out;  and  4)  Inte¬ 
gration  —  the  product  becomes  a  part  of 
Army  training 

•  ACTIONS  TO  BE  TAKEN:  Use  the 

strategies  selected  on  Page  1 5  to  deter¬ 
mine  which  actions  should  be  taken  at  each 
of  the  four  stages  For  example,  if  education 
is  a  solution  to  your  problem  go  to  Page  27 
and  decide  what  sort  of  education  is 
necessary  to  orient  the  user.  Go  to  Page  35 
to  decide  what  sort  of  education  is  necessary 
in  fielding.  Go  to  Page  43  to  decide  what 
sort  of  education  is  necessary  for  trial.  Go  to 
Page  51  to  decide  what  sort  of  education 
is  necessary  for  integration. 


.  RESPONSIBILITIES  OF  DEVELOPERS, 
USERS  AND  SUPPORTERS:  When  you 
know  what  actions  to  take  at  each  stage, 
determine  which  agencies  should  make  the 
necessary  preparation  and  take  that  action. 
For  example,  Page  25  provides  a  worksheet 
for  planning  roles  and  responsibilities  during 
the  Orientation  Stage.  Page  33  is  for 
Fielding,  Page  41  for  Trial,  and  Page  49 
for  Integration. 

•  DESIRED  OUTCOMES:  Use  the  results  of 
the  training  product  analysis  (Page  9)  to 
specify  the  desired  outcomes  which  will 
indicate  success  at  each  stage  in  the 
Implementation  Program.  Page  29  provides 
a  worksheet  for  specifying  outcomes  for  the 
Orientation  Stage.  Page  37  is  for  Fielding, 
Page  45  for  Trial,  and  Page  53  for 
Integration. 


FIGURE  7 

DEVELOP  A  PLAN  FOR 

THE  IMPLEMENTATION  PROGRAM 


Use  appropriate  pages  of  this  guide  to 
translate  the  product  analysis  (pg.  9)  and 
the  selected  implementation  strategy(ies) 
(pg.  15)  into  a  plan  for  the  implementation 
program. 


Block  1 :  Implementation  Analysis  Block  2:  Implementation  Planning 


Conduct 
Training 
product 
analysis 

You  are  here 


Select 

Implementation 

strategy 


Develop 
a  plan  for 
Implementation 
Program 


Preparation  Execution  Success 

Indicators 


IMPLEMENTATION 

ANALTS18 


t  M  PL£  M  E  NT  ATION 
PLANNING 


IMPLEMENTATION 


MON  I  TONING 
ANO  FEEDBACK 


IMPLEMENTATION  PLANNING 

DEVELOP  A  PLAN  FOR 
MONITORING  AND  FEEDBACK 

STEP  3 


FIGURE  8 

WORKSHEET 

FOR  DEVELOPING  A  PLAN 

FOR  MONITORING  AND  FEEDBACK 


BLOCK  3 

THE  IMPLEMENTATION  PROGRAM 


The  Implementation  Program  tracks  a 
product  from  its  production  and  deployment 
to  obsolescence.  The  program  is  composed  of 
four  stages,  and  each  stage  has  three  phases, 
as  is  shown  in  Figure  9. 

The  stages  consist  of: 

•  ORIENTATION:  The  user  learns  about  the 
product  which  is  coming 

•  FIELDING:  The  product  is  delivered  and 
the  user  prepares  to  implement 

•  TRIAL:  Users  receive  support  from  external 
agencies  (e.g  TRADOC,  DARCOM)  in  using 
the  product 

•  INTEGRATION:  The  product  becomes  a 
standard  part  of  training  in  units 


The  three  phases  in  each  stage  are: 

•  PREPARATION:  Agreements  are  reached 
and  arrangements  are  made  to  introduce 
the  product  provide  implementation 
support  and  ensure  that  the  product  gets 
integrated. 

•  EXECUTION:  Actions  are  carried  out  by 
agencies  external  to  the  receiving  unit  to 
introduce  the  product  and  support  its  use. 

•  SUCCESS  INDICATORS:  Steps  are  taken 
by  users  to  learn  about  the  product  to  use 
it  and  finally  to  integrate  it  into  their  philoso¬ 
phy  and  training  practice. 

By  now  you  have  filled  out  all  the  work¬ 
sheets  for  the  Implementation  Program 
(Worksheets  from  pages  25-53,  completed 
during  step  2  of  Implementation  Planning). 
These  sheets  can  now  be  used  to  guide  the 
implementation  effort 


FIGURE  9 

BLOCK  3 

IMPLEMENTATION  PROGRAM 


BLOCK  3 

IMPLEMENTATION  PROGRAM 


PREPARATION  EXECUTION  SUCCESS  INDICATORS 


IMPLEMENTATION  PROGRAM 

ORIENTATION 

STAGE  1 


The  Orientation  Stage  begins  when  a 
product  reaches  the  production  and  deploy¬ 
ment  phase  in  the  Army's  Life  Cycle  System 
Management  Model  (LCSMM).  At  this  point 
receiving  units  need  to  be  informed  about  the 
new  product  which  is  coming  Orientation  has 
three  phases  shown  in  Figure  1 0: 

•  PREPARATION:  External  Agencies 
Formally  Agree  to  Support  Implementa¬ 
tion  and  Prepare  for  Fielding.  The 

appropriate  agencies  need  to  develop  a 
basis  of  issue  plan,  delivery  schedules  for 
the  product  and  an  implementation 
support  plan.  Then  all  the  external  agencies 
involved  in  implementation  need  to  agree 
to  the  schedules  and  implementation  plan. 

•  EXECUTION:  Orientthe  User.  Successful 
orientation  communicates  the  products 
expected  utility,  informs  users  about  the 
product's  overhead  and  skill  requirements, 
provides  technical  and  financial  aid  (if 
necessary)  to  prepare  user  for  the  product 
and  coordinates  the  user's  implementation 
preparations 

•  SUCCESS  INDICATOR:  User  Wantsthe 
Product.  A  user  has  been  successfully 
oriented  when  key  individuals  in  the  chain 
of  command  understand  and  want  the 
product  and  when  the  user  organization 
has  developed  a  utilization  plaa 

Because  lead  time  is  always  a  scarce 
commodity,  Orientation  is  frequently  overlooked 
or  only  partially  done.  Orientation  is  a  real 
opportunity  to  get  a  jump  on  implementation  and 
reduce  a  lot  of  misunderstanding  and  problems 
later  on. 


FIGURE  10 

ORIENTATION 

STAGE  1 


It 


ORIENTATION 

STAGE  1 

PREPARATION  PHASE 


Before  users  can  be  alerted  that  a  product 
is  on  its  way  external  agencies  must  agree  to 
support  implementation  and  prepare  for  fielding. 
Orientation  will  be  much  more  successful  if 
developers  ensure  that  the  following  steps  are 
taken  before  contacting  the  users: 

•  EXTERNAL  AGENCIES  FORMALLY 
AGREE  TO  FIELDING  AND 
IMPLEMENTATION  SUPPORT  PLANS. 

Agencies  involved  in  implementation 
(eg.,  TRADOC,  DARCOM,  Operational 
MACOMs)  should  agree  up-front  on  every¬ 
one’s  role  throughout  the  implementation 
program.  Frequently  agreements  between 
external  support  agencies  do  not  extend 
beyond  fielding  the  product,  and  long-term 
support  is  overlooked.  It  is  harder  to  get 
people  to  commit  additional  support  for  a 
product  after  it  reaches  the  field.  Formal 
agreements  also  provide  institutional 
memory  that  will  help  the  implementation 
program  weather  personnel  turbulance 
within  the  agencies  involved  in  implement¬ 
ing  and  supporting  the  product. 


•  APPROPRIATE  AGENCIES  DEVELOP  A 
BASIS  OF  ISSUE  PLAN,  DELIVERY 
SCHEDULES  AND  AN  IMPLEMENTA¬ 
TION  SUPPORT  PLAN.  Users  can  begin 
their  planning  when  they  have  a  realistic 
delivery  date  and  have  some  idea  what  the 
implementation  program  is  going  to  look 
like.  This  information  should  be  available 
to  the  user  in  time  to  make  necessary 
arrangements,  but  should  not  arrive  so  early 
that  no  action  is  required  for  weeks  or 
months 

The  information  which  you  developed  for 
the  worksheet  shown  in  Figure  1 1  can  be  used 
to  guide  this  phase  of  Orientation. 


FIGURE  11 

WORKSHEET  FOR  PLANNING  AND 
GUIDING  PREPARATION  PHASE 

STAGE  1:  ORIENTATION 


FIGURE  12 

WORKSHEET  FOR  PLANNING 
AND  GUIDING  EXECUTION  PHASE 

STAGE  1:  ORIENTATION 


TRADOC  role:  Communicates  training  concepts 
and  olan  for  hand-off 


OARCOM/ARRCOM  role:  Communicates  plans 
for  training  materiel  if  any 


ORIENTATION 

STAGE  1 : 

SUCCESS  INDICATOR  PHASE 


ItfPlfMfNTAnOM 


«»UMlNUnON 


ACCEPTANCE: 
USERS  PERCEIVE  A 
NEED  FOR  PRODUCT 


KNOW-HOW: 

USERS  ARE  AWARE 
OF  PRODUCTS 
OVERHEAD  AND  SKILL 
REQUIREMENTS 


MANAGEMENT: 

USER  ORGANIZATION 
HAS  PLAN  TO  MANAGE 
PRODUCTS  OVERHEAD 


POLICY: 

USER’S  IMPLEMENTA¬ 
TION  CONCEPT  IS  IN 
ACCORD  WITH 
DEVELOPER’S  CONCEPT 
OF  USE 


Orientation  is  successful  if  the  user  wants 
the  product  Success  is  indicated  when  key 
individuals  in  the  chain  of  command  understand 
and  want  the  product  and  when  the  user 
organization  has  a  utilization  plan. 


KEY  INDIVIDUALS  are  successfully  oriented 
when  they; 

•  PERCEIVE  A  NEED  FOR  THE  PRODUCT 

•  ARE  AWARE  OF  THE  PRODUCTS  OVER¬ 
HEAD  AND  SKILL  REQUIREMENTS 


The  USER  ORGANIZATION  is  oriented  when 

it 

•  HASA  PLAN  TO  MANAGETHE  PRODUCTS 
OVERHEAD 

•  HASAN  IMPLEMENTATION  CONCEPT  IN 
ACCORD  WITH  THE  DEVELOPER’S 
CONCEPT  OF  USE 

If  the  above  conditions  are  met  the  user 
is  ready  to  receive  the  product  This  readiness 
is  perishable  and  will  degrade  quickly  if  there  are 
too  many  delays  between  orientation  and  fielding. 

Information  which  you  developed  for  the 
worksheet  shown  in  Figure  1 3  establishes  criteria 
for  successful  orientation. 


FIGURE  13 

WORKSHEET  FOR  ESTABLISHING 
SUCCESS  INDICATORS 

STAGE  1:  ORIENTATION 


USER  WANTS  THE  PRODUCT 


KNOW-HOW: 

Users  are  aware  of  produces  overhead  and  skill 
requirements 


•  Training  managers: 


•  Trainers 


•  Maintenance  personnel: 


ACCEPTANCE: 

Users  perceive  a  need  for  product 


•  Training  managers; 


•  Trainers 


•  Trainees 


MANAGEMENT: 

Users  Organization  has  plan  to  manage  product's 
overhead 


•  For  supporting  use: 


POLICY: 

Users  implementation  concept  is  in  accord  with 
developer's  concept  of  use 


•  Utilization  concept: 


•  For  controlling  and  maintaining: 


•  Enforcement  concept 


FIELDING 

STAGE  2 


The  Fielding  stage  includes  all  the  efforts 
to  deliver  equipment  documentation,  training 
and  logistic  support  As  was  the  case  for 
Orientation,  Fielding  has  three  phases  as  is 
shown  in  Figure  14. 

•  PREPARATION:  User  formally  agrees 
to  support  product  implementation.  In 

this  phase  agreements  and  plans  developed 
during  Orientation  are  formalized.  The  user 
signs  fielding  agreements  and  memoranda 
of  understanding  regarding  external  imple¬ 
mentation  support  The  user  prepares 
facilities  and  personnel  as  necessary  to 
receive  the  product 

•  EXECUTION:  Field  the  product  and 
provide  fielding  support.  In  addition 

to  delivering  any  hardware  or  software,  a 
successful  fielding  program  publicizes 
demonstrations  of  the  product’s  utility, 
(e.g.,  Operational  Test(OT)  results),  teaches 
users  how  to  use  the  product  supports 
fielding  with  outside  resources,  and  reviews 
local  implementation  plana 


•  SUCCESS  INDICATOR:  User  arranges 
to  use  the  product.  A  product  has  been 
successfully  fielded  when  the  user  intends 
to  use  the  product  and  has  learned  how  to 
use  it  and  when  the  user  organization 
has  assigned  roles  and  resources  for  initial 
use  and  has  established  implementation 
schedules 

Fielding  is  more  than  product  delivery. 
Successful  implementation  requires  that  every 
effort  be  made  to  see  that  receipt  and  early  use 
of  the  product  puts  no  unnecessary  burden  on 
the  user. 
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AND  PERSONNEL  TO 
RECEIVE  PRODUCT 


The  fielding  (and  later  implementation) 
efforts  will  be  more  successful  if  the  user 
formally  agrees  to  support  product  implemen¬ 
tation  in  advance  of  actual  fielding. 


USER  AGREES  TO  ACCEPT  THE 
PRODUCT  AND  THE  EXTERNAL 
IMPLEMENTATION  SUPPORT.  Users 
make  a  formal  commitment  to  provide  the 
resources  necessary  to  receive  the  product 
and  participate  in  the  implementation 
program.  Without  formal  negotiation  and 
agreement  in  advance  of  delivery,  you  may 
discover  too  late  that  users  have  plans  for 
the  product  which  differ  greatly  from  yours 
or  that  the  pressures  of  the  moment  make 
the  users  want  to  cut  back  on  the  support 
that  is  needed  to  carry  out  the  implemen¬ 
tation  program. 


•  USER  PREPARES  FACILITIES  AND 
PERSONNEL  TO  RECEIVE  PRODUCT. 

Any  special  preparation  of  storage  or 
classroom  facilities  equipment  or  personnel 
should  be  arranged  and  checked  on  in 
advance  of  fielding. 

Getting  user  agreements  to  support  product 
implementation  can  reduce  many  headaches 
in  the  field.  It  is  similar  to  making  reservations 
in  advance,  so  there  are  no  surprises  for  either 
the  user  or  the  fielding  team. 

The  information  which  you  developed  for 
the  worksheet  shown  in  Figure  1 5  can  be  used 
to  guide  this  phase  of  Fielding 


FIGURE  15 
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The  fielding  effort  gets  the  user  set  to  use 
the  product  In  addition  to  product  delivery, 
TRADOC  may  hand-off  a  complete  training 
support  package,  DARCOM  may  provide  new 
equipment  training  and  equipment  assistance, 
and  User  MACOM’s  may  be  involved  in  coordinating 
and  monitoring  implementation. 

•  MOTIVATE:  Arrange  demonstrations  of 
product  utility  and  publicize  favorable 
results.  Show  users  what  the  product  can 
do  for  them  so  that  they  will  become 
motivated  to  use  the  product  themselves 

•  EDUCATE:  Teach  users  how  to  use 
product.  Through  Mobile  Training  Teams 
workshops  or  assisted  on-the-job-training 
ensure  that  personnel  have  mastered  the 
skills  and  required  knowledge  to  enable 
proper  use  of  the  product 


•  ASSIST:  Support  fielding  with  outside 
resources.  Provide  technical  and  logistic 
support  and  training  assistance  to  help 
the  user  receive  the  product  and  make 
arrangements  for  using  it 

•  REGULATE:  Review  local. implemen¬ 
tation  plans.  Make  sure  that  the  receiving 
unit  has  published  directives  which  assign 
roles  and  resources  for  initial  use  of  the 
product  and  establish  schedule  for  use. 

The  information  which  you  develop  for  the 
worksheet  shown  in  Figure  1 6  can  be  used 
to  guide  the  execution  phase  of  Fielding 
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FIGURE  16 

WORKSHEET  FOR  PLANNING  AND 
GUIDING  THE  EXECUTION  PHASE 

STAGE  2:  FIELDING 


MOTIVATE:  Arrange 
demonstrations  of  pro¬ 
duct's  utility  and  EDUCATE:  Teach  users  ASSIST:  Support  fielding  REGULATE:  Review  local 

publicize  favorable  results  how  to  use  product  with  outside  resources  implementation  plans 

(e.g.  OT  or  validation 

test  results) 


DARCOM/ARRCOM  role: 


Fielding  is  successful  when  the  user  is 
prepared  to  use  the  product  Success  is 
indicated  if  individuals  user  can  and  will  try 
the  product  and  when  the  user  organization 
has  a  utilization  program. 

Individual  users  are  successfully  oriented 
when  they; 

•  INTEND  TO  USE  THE  PRODUCT 

.  HAVE  LEARNED  HOW  TO  USE  THE 
PRODUCT 


The  user  organization  is  successfully 
oriented  when  it 

•  HAS  ASSIGNED  ROLES  AND  RESOURCES 
FOR  INITIAL  USE  OF  PRODUCT 

•  HAS  ESTABLISHED  IMPLEMENTATION 
SCHEDULES 


Successful  fielding  means  more  than 
delivering  the  product;  it  means  preparing 
personnel  and  units  to  continue  using  the 
product  on  their  own,  without  the  extra  support 
that  often  accompanies  fielding  efforts 

The  information  which  you  developed  for 
the  work  sheet  shown  is  Figure  1 7  establishes 
the  criteria  for  success  in  the  fielding  stage. 


FIGURE  17 

WORKSHEET  FOR  ESTABLISHING 
SUCCESS  INDICATORS 

STAGE  2:  FIELDING 
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STAGE  3 


The  Trial  Stage  begins  when  users  start 
using  the  product  on  their  own.  The  stage  ends 
when  the  users  have  dealt  with  all  the  start¬ 
up  problems  and  have  decided  to  institutionalize 
use  of  the  product  or  to  discontinue  use.  Trial 
has  three  phases  as  is  shown  in  Figure  1 8. 

•  PREPARATION:  External  agencies 
arrange  to  support  user  trial.  If  external 
support  must  be  continued  past  fielding, 
appropriate  agencies  must  prepare 
materials,  programs,  and  procedures  to 
support  user  trial.  Appropriate  agencies 
must  also  develop  product  evaluation  plans 
to  track  product  utilization  and  effectiveness, 

•  EXECUTION:  Support  user  trial.  A 

successful  trial  effort  solicits  and  publicizes 
product  success  stories,  obtains  and 
communicates  lessons- learned  about  using 
the  product  and  provides  resources  and 
technical  assistance  to  support  initial  use 
of  the  product  Evaluation  teams  will  also 
monitor  use  and  evaluate  effectiveness  of 
the  product  during  the  trial  stage. 

•  SUCCESS  INDICATOR:  User  intends  to 
sustain  the  product.  A  product  has  had 
a  successful  trial  when  individual  users 
are  satisfied  with  the  product  and  when 
the  user  organization  has  developed  a 
sustainment  program. 

Most  product  die  in  the  trial  stage  if  they 
are  going  to  die  at  all.  Users  either  never  begin 
using  the  product  or  are  overwhelmed  with  start¬ 
up  problems  for  which  they  lack  the  personnel, 
time  or  resources  to  solve. 


FIGURE  18 
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Most  products  succeed  or  fail  in  the  Trial 
Stage.  Products  always  cause  more  problems 
than  anyone  expects  (that's  Murphy's  Law), 
and  users  may  simply  give  up  on  a  product 
when  they  encounter  those  unexpected  pro¬ 
blems  To  forestall  that  possibility,  all 
appropriate  agencies  should  be  prepared  to  take 
steps  to  ensure  that  the  product  gets  a  fair 
trial  in  the  user  environment 

•  USERS  AGREE  TO  SUPPORT  PRODUCT 
EVALUATION.  Users  should  be  pre¬ 
pared  to  provide  feedback  on  problems 
as  well  as  successes  they  are  having  during 
early  use  of  the  product  This  feedback 
will  help  external  agencies  to  adjust 
implementation  support  as  necessary  and 
will  provide  input  to  the  user's  own 
decisions  about  sustaining  or  discounting 
use  of  the  product 


This  information  which  you  developed  for 
the  worksheet  shown  in  Figure  19  can  be  used 
to  guide  this  phase  of  the  Trial  Stage. 


•  EXTERNAL  AGENCIES  ARRANGE  TO 
SUPPORT  USER  TRIAL  AND  EVALUATE 
PRODUCT.  I  n  order  to  make  timely  delivery 
of  implementation  support  external 
agencies  must  prepare  materials  and 
assistance  programs  well  in  advance  of  the 
time  they  are  needed  in  the  user  environ¬ 
ment  In  addition,  plans  must  be  developed 
for  obtaining  data  from  the  user  to  evaluate 
the  product  and  contribute  to  decisions 
about  life-cycle  support 


FIGURE  19 

WORKSHEET  FOR  PLANNING  AND 
GUIDING  PREPARATION  PHASE 

STAGE  3:  TRIAL 


TRIAL 

STAGE  3 

EXECUTION  PHASE 


The  effort  to  support  user  trials  is  intended 
to  ensure  that  use  of  the  product  is  a  go  in 
the  operational  environment  The  major 
complaint  of  receiving  units  is  that  products 
get  dumped  on  them.  Developers  arrive  with 
much  fanfare,  field  a  product  and  disappear, 
leaving  the  user  alone  to  solve  all  the  start¬ 
up  problems  To  counter  these  problems  and 
to  prepare  the  user  to  sustain  the  product 
provide  support  to  user  trial. 

•  MOTIVATE:  Solicit  and  publicize  any  •  REGULATE:  Monitor  use  and  evaluate 

product  success  stories.  Tell  people  effectiveness  of  product.  Ensure  that 

when  their  peers  are  getting  better  ratings  the  user  command  and  the  external 

using  less  resources  or  getting  more  evaluation  agencies  are  directed  to  report 

training  done  by  using  the  product  data  concerning  product  use.  The  data 

should  represent  frequency  and  quality  of 

•  EDUCATE:  Obtain  and  communicate  use,  resources  consumed,  and. training 

lessons  learned  about  using  product.  effectiveness 

Circulate  commanders’  comments  user's 

guides  etc  so  that  all  users  benefit  from  The  information  which  you  developed  for 

the  experiences  others  are  having  with  the  worksheet  shown  in  Figure  20  can  be 
the  product  used  to  guide  the  Execution  phase  of  the  Trial 

Stage. 

•  ASSIST:  Provide  resources  and  technical 
assistance  to  support  initial  use  of 
product.  This  assistance  will  help  users 
to  solve  start-up  problems  Depending  on 
the  product  this  could  be  ammunition, 
technical  experts  contract  maintenance, 
etc,  but  do  not  over  commit  yourself  or 
raise  false  expectations  among  users 
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FIGURE  20 

WORKSHEET  FOR  PLANNING  AND 
GUIDING  THE  EXECUTION  PHASE 

STAGE  3:  TRIAL 
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TRADOC  role:  Monitor,  evaluate,  and  provide  feedback  about  use  of  product 


TRIAL 

STAGE  3 

SUCCESS  INDICATOR  PHASE 


Trial  is  a  success  if  the  user  intends  to 
sustain  the  product  Success  is  indicated  if 
individuals  a.e  satisfied  with  the  product  and 
when  the  user  organization  has  a  sustainment 
program. 

Individual  users  have  had  a  successful  trial  if  The  user  organization  has  given  the  product  a 
they;  successful  trial  if: 

•  LIKE  THE  PRODUCT  .  UNIT  COMMANDERS  HAVE  MADE 

DECISIONS  TO  SUSTAIN  THE  USE  OF 
.  ARE  EXPERIENCED  IN  PRODUCT  USE  THE  PRODUCT 

•  THE  USER  ORGANIZATION  HAS  A 
PRODUCT  UTILIZATION  POLICY 

By  carrying  the  implementation  program 
through  initial  user  trials,  you  can  help  ensure 
that  your  product  will  not  be  one  of  those  which 
end  up  on  storage  shelves  because  it  proves 
too  difficult  or  troublesome  to  use. 

The  information  which  you  developed  for 
the  worksheet  shown  in  Figure  21  establishes 
criteria  for  success  during  the  Trial  Stage. 


FIGURE  21 

WORKSHEET  FOR  ESTABLISHING 
SUCCESS  INDICATORS 

STAGE  3:  TRIAL 


USER  INTENDS 
TO  SUSTAIN  PRODUCT 


ACCEPTANCE: 

Users  like  using  the  product 


1.  Training  managers: 


2.  Trainers: 


3.  Trainees: 


MANAGEMENT: 

Unit  commanders  have  made  decisions  to  sustain 
use  of  product 


1.  Administrative  and  logistic  levels: 


KNOW-HOW: 

Key  users  are  experienced  In  product  use 


1  Training  managers: 


2.  Trainers: 


3.  Trainees: 


POLICY: 

User  organization  has  product  utilization  policy 


1.  To  use: 


2,  Operational  user  level: 


2.  To  monitor  use: 


INTEGRATION 

STAGE  4 


The  Integration  stage  begins  when  users 
decide  to  make  sustained  use  of  the  product 
Integration  has  three  phases  as  is  shown  in 
Figure  22. 

•  PREPARATION:  External  support 
agencies  agree  to  support  product 
sustainment.  If  the  product  is  to  become  a 
permanent  part  of  training  in  units,  external 
support  agencies  need  to  agree  to  integrate 
the  product  into  training  and  maintenance 
literature,  the  schools,  procurement  pro¬ 
grams,  etc 

•  EXECUTION:  Integrate  the  product  into 
external  support  systems.  A  successful 
integration  effort  will  include  product 
information  in  all  relevant  training  and 
maintenance  literature,  integrate  the 
product  into  TRADOC-conducted  resident 
training  and  integrate  product  support  into 
routine  procedures  for  supporting  opera¬ 
tional  units. 


•  SUCCESS  INDICATOR:  User  integrates 
product  into  training  in  units.  A  product 
is  successfully  integrated  when  individuals 
and  or  units  make  routine  use  of  the 
product,  and  when  the  user  organizations 
integrate  the  product  into  routine  training 
practices 

The  integration  of  the  product  into  training 
continues  with  product  modifications  updates 
in  support,  documentation,  etc  throughout  the 
life-cycle. 


FIGURE  22 

INTEGRATION 

STAGE  4 
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INTEGRATION 

STAGE  4 

PREPARATION  PHASE 


Even  after  a  successful  trial,  a  new  product 
may  cease  to  be  used  if  its  use  is  in  conflict 
with  other  elements  of  the  training  system  and 
the  training  support  system  of  the  Army. 

These  conflicts  will  be  minimized  when 
external  support  agencies  make  formal 
agreements  to  support  product  sustainment 

•  EXTERNAL  AGENCIES  AGREE  TO 
INTEGRATE  PRODUCT  INTO  TRAINING 
AND  MAINTENANCE  LITERATURE 
AND  TRAINING  SUPPORT  PRO¬ 
CEDURES.  A  product  may  be  overlooked 
or  neglected  unless  people  can  quickly 
and  easily  find  out  how  to  use,  maintain 
and  support  it  Integrating  the  product  into 
the  literature  insures  that  use  and  sustain¬ 
ment  guidance  is  widely  known  and  easily 
accessible. 


•  EXTERNAL  SUPPORT  AGENCIES 
PREPARE  TO  DELIVERY  LONG-TERM 
SUPPORT  TO  USER  UNITS.  If  units 
cannot  support  a  product  given  their 
available  resources,  they  must  have  some 
help.  This  assistance  should  be  pro¬ 
grammed  as  soon  as  user  trials  have 
indicated  what  long-term  support  is  required. 

The  information  which  you  developed  for 
the  worksheet  shown  in  Figure  23  can  be  used 
to  guide  this  phase  of  the  Integration  Stage. 
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FIGURE  23 

WORKSHEET  FOR  PLANNING  AND 
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If  the  product  is  to  become  a  standard  part 
of  US  Army  training  then  it  must  be  integrated 
into  the  support  systems  of  agencies  external  to 
operational  units  (TRADOC,  ARRCOM.  opera¬ 
tional  MACOMs).  Otherwise  product  use  may 
wither  away  for  lack  of  resources  or  incentive. 

•  MOTIVATE:  Include  product  information 
in  all  relevant  training  and  maintenance 
literature.  If  soldiers  and  leaders  are 
constantly  confronted  with  information 
about  the  product  they  will  come  to  see  it 
as  the  way  to  train. 

•  EDUCATE:  Integrate  product  into 
TRADOC-conducted  resident  training  as 
appropriate.  Ensure  that  trainers  and 
trainees  go  to  their  units  prepared  to  use 
the  product  and,  when  appropriate,  provide 
refresher  training  in  units 


•  ASSIST:  Integrate  product  support  into 
routine  procedures  for  supporting 
operational  units.  Ensure  that  amounts  of 
ammunition,  supplies  and  spare  parts 
required  by  the  product  are  procured  and 
made  available  to  units  on  a  routine  basis 

•  REGULATE:  Army  and  MACOM  regula¬ 
tions  establish  guidelines  for  product 
use.  Formal  inspection  procedures  should 
include  checks  on  product  usage. 

The  information  which  you  developed  for 
the  worksheet  shown  in  Figure  24  can  be  used 
to  guide  the  Execution  Phase  of  the  Integra¬ 
tion  Stage. 
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TRAINING 


MANAGEMENT: 
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MANAGING  PRODUCT 
OVERHEAD  ARE 
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STANDARD  MANAGE¬ 
MENT  PROCEDURES 


POLICY: 

USER  ORGANIZATION 
ENFORCES  UTILIZATION 
POLICY 


Integration  of  the  product  into  Army  training 
is  successful  when  Individuals  make  routine  use 
of  the  product  and  when  the  user  organization 
integrates  the  product  in  training  practices 

Individual  Users  have  successfully  integrated 
the  product  into  training  when  they; 

.  ACCEPT  PRODUCT  AS  A  NORMAL  PART 
OF  TRAINING 

•  HAVE  THE  KNOW-HOW  TO  USE  THE 
PRODUCT  IN  ROUTINE  TRAINING 


The  User  Organization  has  successfully  inte¬ 
grated  the  product  into  training  when: 

•  PROCEDURES  FOR  MANAGING 
PRODUCT  OVERHEAD  ARE  INTEGRATED 
INTO  STANDARD  MANAGEMENT 
PROCEDURES 

•  IT  ENFORCES  ITS  UTILIZATION  POLICY 


The  indicators  of  success  in  Stage  4 
(Integration)  represent  solution  of  implementation 
problems  —  Acceptance,  Know-How,  Manage¬ 
ment  and  Policy  —  diagnosed  in  Step  2  of  the 
Implementation  Analysis.  These  indicators,  if 
monitored  at  intervals,  will  track  long-term 
support  needs  and  product  improvement  which 
result  from  the  regular  use  of  the  product 
The  information  which  you  developed  for 
the  worksheet  shown  in  Figure  25  establishes 
criteria  for  successful  integration  of  the 
product  into  Army  training  procedures 


FIGURE  25 

WORKSHEET  FOR  ESTABLISHING 
SUCCESS  INDICATORS 

STAGE  4:  INTEGRATION 


These  indicators  should  — —  -  -  — - 

represent  the  product  being  USER  INTEGRATES  PRODUCT 
used  as  intended  in  the  INTO  TRAINING  IN  UNITS 

user  environment. _ 


ACCEPTANCE: 

Personnel  accept  product  as  a  normal  part  of 
training 


1  Training  managers: 


2.  Trainers: 


3.  Trainees: 


MANAGEMENT: 

Procedures  for  managing  product  overhead  are 
integrated  into  standard  management  procedures 


1.  To  manage  overhead  for  using  product  (e  g.  con¬ 
sumables.  time,  ranges,  etc.): 


2.  To  manage  overhead  for  controlling  and  maintaining 
product: 


KNOW-HOW: 

Personnel  know-how  to  use  product  in 
routine  training 


1.  Informed  training  managers: 


2.  Prepared  trainers: 


3.  Prepare  trainees: 


POLICY: 

Utilization  policy  is  enforced 


1.  Frequency  of  use  prescribed: 


2.  Quality  of  use  attained: 


3.  Frequency  of  inspection: 


BLOCK  4 


The  overview  map  shown  at  the  top 
of  this  page  has  been  used  to  track  progress 
through  this  guide.  It  shows  monitoring  and 
feedback  as  occurring  after  everything  has 
been  done.  The  fact  is,  however,  that 
monitoring  and  feedback  occurs  at  each 
phase  of  every  stage  in  the  Implementation 
Program.  The  plan  for  monitoring  and  feed¬ 
back  was  developed  during  Step  3  of  Imple¬ 
mentation  Planning  (Block  2).  The  steps 
described  on  the  following  pages  of  this 
guide  are  for  collecting  the  feedback  informa¬ 
tion  during  each  phase  of  the  Implementation 
Program  (Block  3),  diagnosing  problems, 
deciding  who  should  solve  them,  and  then 
feeding  relevant  information  back  to  appro¬ 
priate  action  agencies. 


The  purpose  of  Monitoring  and  Feedback  is 
to  see  that  things  are  being  done  right ,  that 
the  right  things  are  being  done,  and  that  the 
right  people  find  out  about  it  There  are  three 
steps  in  monitoring  and  feedback  as  shown  in 
Figure  26. 

STEP  1:  MONITOR  IMPLEMENTATION. 

Collect  data  to  describe  the  current  status 
of  the  implementation  program:  prepara¬ 
tion,  execution  and  successes 

STEP  2:  DIAGNOSE  PROBLEMS  AND 
ACHIEVEMENTS.  Analyze  the  data 
collected  in  Step  1  to  determine  if  the 
preparations  were  adequate,  the  program 
effectively  executed,  and  success  achieved 
at  each  stage  of  implementation. 

STEP  3:  PROVIDE  FEEDBACK  TO  APPRO¬ 
PRIATE  ACTION  AGENCIES.  I  nformation 
is  fed  back  to  facilitate  negotiations  and 
decision  making  to  improve  the  product 
or  its  support,  as  necessary. 
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FIGURE  26 
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MONITORING  AND  FEEDBACK 
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MONITORING  AND  FEEDBACK 
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STEP  2 


STEP  3 


MONITORING  AND  FEEDBACK 

MONITOR  IMPLEMENTATION 

STEP  1 
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Monitor  implementation  in  order  to  collect 
timely  information  about  the  Preparation,  Execu¬ 
tion  and  Success  Indicators  at  each  stage  of 
the  Implementation  Program. 

•  MONITOR  PREPARATIONS  MADE  IN 
EACH  STAGE:  Although  you  may  not  be 
able  to  observe  the  actions  of  other  con¬ 
cerned  agencies,  you  can  keep  in  touch 
with  the  preparations  through  telephone 
calls,  having  yourself  placed  on  agency 
information  routing  lists,  and  keeping  in 
touch  with'  points  of  contact  on  a  scheduled 
basis 

•  OBSERVE  EXECUTION  FOR  EACH 
STAGE:  To  ensure  that  things  are  being 
done  right  collect  on-site  information  from 
a  sample  of  units  involved  in 
implementation. 


•  MEASURE  SUCCESS  INDICATORS  IN 
EACH  STAGE:  The  most  important 
information  comes  from  users.  Be 

certain  to  ask  (either  oK arve  or  sur¬ 
vey  individuals  and  uni+  eaders)  if 
the  implementation  progrc  is  having 
the  desired  effect!  If  it  is  not  pos¬ 
sible  to  obtain  information  from  every 
user  unit,  be  sure  to  obtain  informa¬ 
tion  from  a  sample  that  is  representa¬ 
tive  of  the  different  kinds  and  loca¬ 
tions  of  user  units. 

Next  to  the  execution  of  implementa¬ 
tion  program,  monitoring  implementation 
is  the  most  important  action  to  take.  You 
have  to  know  what  impact  your  program:  is 
having  on  users  in  order  to  determine  what 
actions  to  take  next. 

Different  products  may  require  differ¬ 
ent  kinds  of  data  recording  forms  for  col¬ 
lecting  feedback.  An  example  of  a  possi¬ 
ble  data  recording  form  is  shown  in  Figure 
27.  You'll  need  forms  to  record  informa¬ 
tion  about  each  phase  of  each  stage. 
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MONITORING  AND  FEEDBACK 

DIAGNOSE  PROBLEMS 
AND  ACHIEVEMENTS 

STEP  2 


This  step  in  monitoring  and  feedback  is 
designed  to  diagnose  strengths  and  weak¬ 
nesses  in  the  implementation  program.  It  requires 
the  data  obtained  about  EEA  (which  were 
developed  during  Step  3  of  implementation 
planning;  See  Pg.  19)  be  analyzed  and  results 
quantified.  Quantifying  serves  two  purposes 
First  it  reduces  results  to  the  simplest  possible 
terms  If,  for  example,  you  use  a  scale  that 
ranges  from  0-4  where  0  indicates  “totally  un¬ 
satisfactory”  for  a  given  element  of  the  program 
and  4  indicates  “totally  satisfactory",  then  the 
current  status  on  any  particular  element  of  the 
program  can  be  easily  translated  into  a  need  for 
action:  0  or  1  means  “do  something  to  improve 
the  situation  with  respect  to  this  element”,  2 


means  “keep  a  careful  eye  on  this  element", 
and  3  or  4  means  “no  problem”.  The  second 
advantage  of  quantifying  is  that  results  can  be 
easily  scanned  to  identify  the  pattern  of 
implementation  problem  that  is  being  en¬ 
countered.  For  example,  problems  may  cluster 
within  a  particular  stage  (e.g  everything  was 
going  great  until  “triaf’)  or  within  a  particular 
phase  across  stages  (e.g.  success  is  not  indi¬ 
cated  despite  excellent  execution  of  the 
program).  Different  patterns  will  require  different 
kinds  of  corrective  action.  To  complete  Step  2 
you  will  need  a  data  analysis  sheet  for  each 
stage  of  the  implementation  program.  Figure  28 
provides  a  possible  format  for  data  analysis 
sheeta 


FIGURE  28 

AN  EXAMPLE  OF  A 
POSSIBLE  FORMAT  FOR 
DATA  ANALYSIS  SHEETS 


Using  “current  status”  information 
(see  Pg.  57),  answer  these  questions 


DIAGNOSE  PROBLEMS 
AND  ACHIEVEMENTS 


INTEGRATION  STAGE 


TRIAL  STAGE 


FIELDING  STAGE 


ORIENTATION  STAGE 

ARE  PREPARATIONS  ADEQUATE 
TO  SUPPORT  PROGRAM? 


(list  all  EEA 
from  appropriate 
data  collection  forms) 

1 .  Agreements  and 
coordinations 


2.  Plans  and  material 
preparations 


INTEGRATION  STAGE 


TRIAL  STAGE 


FIELDING  STAGE 


(list  all  EEA  from 

appropriate 

data  collection  forms) 

1  Acceptance 

2  Know-how 

3.  Management 


INTEGRATION  STAG 


TRIAL  STAGE 


FIELDING  STAGE 


ORIENTATION  STAGE 


IS  PROGRAM  BEING 
EXECUTED  EFFECTIVELY? 


(list  all  EEA 
from  appropriate 
data  collection  forms) 

1.  Motivate 

2.  Educate 

3.  Assist 


4  Regulate 


4  Policy 


MONITORING  AND  FEEDBACK 

PROVIDE  FEEDBACK  TO 
APPROPRIATE  ACTION  AGENCIES 

STEP  3 


In  order  to  ensure  success  for  the  Imple¬ 
mentation  Program,  you  must  provide  feedback 
to  appropriate  action  agencies  Use  results 
of  the  diagnosis  of  implementation  problems  and 
achievements  to  develop  three  kinds  of  feedback. 

.  IMMEDIATE  FEEDBACK  REGARDING 
SPECIFIC  PROBLEMS  THAT  SHOULD 
BE  CORRECTED  ON-THE-SPOT.  If 

there  is  an  immediate  solution  to  an 
implementation  problem,  make  appropriate 
recommendations  to  the  action  agency. 
Otherwise  return  to  Step  1  of  Implementa¬ 
tion  Planning  and  alteryourimplementation 
program.  You  will  have  to  select  new 
strategies  to  cope  with  persistent  or 
unexpected  problems 

.  MID-RANGE  FEEDBACK  FOR 

BROADER  PROGRAM  ISSUES.  Use  data 
collected  to  compare  program  execution 
with  program  success  This  comparison  will 
tell  you  if  your  plan  is  working,  if  it  needs 
revising,  or  if  the  effort  can  be  reduced 
without  degrading  implementation. 
Problems  identified  by  this  process  may 
require  solution  at  the  MACOM  level. 


•  LONG-RANGE  FEEDBACK  FOR  POLICY 
ISSUES.  Use  diagnosis  of  implementation 
problems  to  assist  high-level  policy  makers 
in  determining  whether  the  present  product 
implementation  program  should  be  con¬ 
tinued,  modified,  delayed  or  scrubbed. 

The  information  shown  in  Figure  29  provides 
general  guidelines  for  feeding  information  back 
to  control  the  implementation  program,  recom¬ 
mend  improvements  in  the  product  and  contri¬ 
bute  to  policy  decisions  to  ensure  that  the 
product  gets  used  to  best  effect  in  the 
operational  environment 


FIGURE  29 

GENERAL  GUIDELINES 
FOR  FEEDBACK 


PROVIDE  FEEDBACK  TO 
APPROPRIATE  ACTION  AGENCIES 


Using  results  of  the  diagnosis  of  implementation 
problems  and  achievements  (see  Pg.  59), 
develop  these  three  kinds  of  feedback. 


Immediate  feedback  regarding 
specific  problems  that  should  be 
corrected  on-the-spot 


Is  there  an  immediate  solution  to  this 
specific  problem'?  j 

Yes 

No 

List  potential 
solution  and 
feedback  to 
appropriate 
action  agency 

List  problem  and 
feedback  to  Step  1 
of  implementation 
planning:  Select 
Implementation 

Strategies  (Pg.  1 5) 

Mid-range  feedback  regarding 
broader  program  issues. 


Is  success  indicated9 

Yes 

No 

1 

Is 

Program 

Y 

E 

S 

! 

The  right  things  are 
being  done  right 

Proceed  as  planned. 

The  wrong  things 
are  being  done.  Notify 
appropriate  action 
agencies  and  feedback 
to  Step  1  (Pg.  1 5)  of 
Implementation  Planning. 

Execution 

1 

Effective9 

Things  planned  are 

not  necessary  at  this 

Things  are  not 

time.  Notify  appropriate 

being  done 

N 

action  agencies  and 

right.  Notify 

0 

feedback  to  Step  2  of 

appropriate 

Implementation 

action  agencies 

Analysis  (Pg.  1 1 ) 

SUMMARY 


This  guide  describes  a  strategy  for  imple¬ 
menting  training  products  in  operational  units 
It  establishes  procedures  which  guide  an  imple¬ 
mentation  effort  from  the  time  a  product  is  fully 
developed  until  it  is  integrated  into  training  in 
units  Its  purpose  is  to  ensure  that  the  product 
is  used  in  the  way  that  it  should  be  used  Using 
the  guide’s  procedures  you  can  analyze  a  train¬ 
ing  products  characteristics,  forecast  potential 
implementation  problems,  develop  and  execute 
a  tailored-made  implementation  program  and 
monitor  your  progress  throughout  the  imple¬ 
mentation  process  The  guide  does  not  tell  you 
how  to  develop  a  product  although  it  does 
provide  feedback  to  developers  about  needed 
modifications  which  might  improve  the  products 
chances  of  being  successfully  implemented.  It 
is  not  a  manual  for  evaluating  the  training  effect¬ 
iveness  of  a  product  although  it  does  identify 
where  such  evaluations  fit  into  an  implementa¬ 
tion  program.  TRADOC  publications  such  as 
TRADOC  Pamphlet  350-30  (Interservice 
Procedures  for  Instructional  Systems  Develop¬ 
ment  [ISD])  describe  procedures  for  training 
product  development.  Publications  such  as 
TRADOC  Pamphlet  71-8  (Analyzing  Training 
Effectiveness)  describe  procedures  for  planning 
and  conducting  training  product  effectiveness 
evaluations.  What  these  manuals  do  not  tell  you 
is  how  to  get  a  training  product  used.  This  guide 
will  help  close  that  "implementation  gap"  and 
ensure  that  products  are  used  to  best  effect  in 
operational  units. 


PRODUCT  MANAGER’S 
IMPLEMENTATION  CHECKLIST 


The  following  checklist  will  assist  in  the  TARGET 

analysis,  planning,  execution  and  monitoring  of  DATE 

an  Implementation  Program  for  an  exportable 

training  product  - - 


CONDUCT  IMPLEMENTATION 
ANALYSIS 

Obtain  information  for  a  training 
product  analysis  (pg  9) 

Identify  Potential  Implementation 
Problems  (pg.  1 1) 

DEVELOP  AN  IMPLEMENTATION 
PLAN 

Select  Implementation  Strategies 

(pg-  is) 

Develop  action  plans  for  each 
implementation  stage  (Execution 
Phases) 

-Stage  1:  (pg  27) 

-Stage  2:  (pg  35) 

-Stage  3:  (pg  43) 

-Stage  4:  (pg  51) 

Coordinate  with  external  agencies 
involved  in  implementation  effort 
(Preparation  Phases) 

-Stage  1:  (pg.  25) 

-Stage  2:  (pg.  33) 

-Stage  3:  (pg.  41) 

-Stage  4:  (pg.  49) 

Establish  outcome  criteria  for  each 
implementation  stage  (Success 
Indicators  Phases) 

-Stage  1:  (pg  29) 

-Stage  2:  (pg.  37) 

-Stage  3:  (pg.  45) 

-Stage  4:  (pg.  53) 


DEVELOP  PLAN  FOR 
MONITORING  AND  FEEDBACK 


TARGET 


TARGET 


EXECUTE  AND  MONITOR 

THE  IMPLEMENTATION  PROGRAM 

Stage  1 :  Orientation 

-  request  support  and  make 
advanced  coordination  (pg  25) 

-  collect  data  and  provide  feedback 
to  ensure  adequate  preparation 
(P9-  57) 

-  execute  action  plans  (pg.  27) 

-  observe  execution  and  provide 
feedback  to  ensure  effectiveness 
(P9-  57) 

-  measures  success  indicators 
(pg  29.  57) 

-  feedback  information  and 
recommend  corrective  action 
(pg  59,  61) 

Stage  2:  Fielding 

-  request  support  and  make 
advanced  coordination  (pg  33) 

-  collect  data  and  provide  feedback 
to  ensure  adequate  preparation 
(P9-  57) 

-  execute  action  plans  (pg  35) 

-  observe  execution  and  provide 
feedback  to  ensure  effectiveness 
(pg-  5.7) 

-  measure  success  indicators 
(pg-37) 

-  feedback  information  and 
recommend  corrective  action 
(pg  59,  61) 


Stage  3:  Trial 

-  request  support  and  make 
advanced  coordination  (pg  41) 

-  collect  data  and  provide  feedback 
to  ensure  adequate  preparation 
(pg57) 

-  execute  action  plans  (pg  43) 

-  observe  execution  and  provide 
feedback  to  ensure 
effectiveness  (pg  57) 

-  measure  success  indicators 

(pg  53) 

-  feedback  information  and 
recommend  corrective  action 
(pg  59,  61) 

Stage  4:  Integration 

•  request  support  and  make 
advanced  coordination  (pg  49) 

-  collect  data  and  provide  feedback 
to  ensure  adequate  preparation 
(P9-  57) 

-  execute  action  plans  (pg.  51) 

-  observe  execution  and  provide 
feedback  to  ensure  effectiveness 

(pg-  57) 

-  measure  success  indicators 

(pg.  53) 

-  feedback  information  and 
recommend  corrective  action 
(pg.  59,  61) 

-  establish  schedule  and 
procedures  for  periodic  review 
of  product  utilization 
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